3D sensors, in particular lidar, radar and similar systems increasingly become the main source of data used for both manned and unmanned vehicles operating in demanding environments. They are providing data e.g. for obstacle warning systems and for landing aid systems. For operational systems the environment is not a clinically well-defined laboratory but data processing has to cope with all aspects of a natural or man-made environment in real-time. When using 3D sensors for the active support of aerial vehicles or autonomous ground vehicles processing the measured 3D data becomes a central task. The amount of data generated by 3D sensors can be tremendous. Therefore, a fast, intelligent and efficient way of data reduction has to be found. This invention is concerned with data selection and reduction at the very first steps in the evaluation of the sensor data (conducted in an analogue pulse analyzer as a subsystem of the 3D sensor).
Particularly for optical sensors used for obstacle warning and as landing aid in degraded vision environment (DVE), requirements for detection are diverse, sometimes competitive, or even contradictory. Primarily the sensor can work as an obstacle warning system where it is very sensitive to small signals provided by the echo of thin wires in the range from tens of meter to more than 1000 m. For doing this, radiation shall pass through air that might be obscured by fog, clouds, dust etc. The echoes originating from these obscurants must not damage the sensor and ideally should be ignored by the 3D sensor. In addition, the sensor can detect ground in the same distance range mentioned before. As the back reflected intensity decreases with 1/r2 or 1/r3 for bulk targets and for wire targets, respectively, the signal receiver has to cope with signal intensities that cover several orders of magnitude.
There are currently two approaches in common 3D measurement systems based on photon runtime measurement, i.e. time of flight counters. The first known art is to record the complete returned intensity sequence, digitize and analyze it digitally in real time. This evaluation requires enormous computing power due to the huge amount of data points on the one hand and the enormous signal dynamics on the other hand ([MUR13, ULL05a, ULL05b]).
Another known approach is to extract discrete points of time by applying a trigger condition on the signal train. The invention as described in the following will apply this particular approach.
A schematic representation of a system used for carrying out such a method is depicted in FIG. 1. An emitter 11 sends pulses of radiation 15 (e.g. laser or radar pulses). Part of the intensity is scattered back or reflected back by an object 16 (e.g. ground, wires, obscurants) and is received by a receiver 12. The signal intensity train or signal amplitude train (short signal train) is investigated in a pulse analyzer 13 producing one or a series of measurement distances which are prepared for further processing 14.
FIG. 2 shows an example of a signal train 21 as received by a receiver 12. The signal train includes several signal peaks evoked by scattering of the emitted pulse by objects in different distances. The intensity of the returning echo pulses is compared to a trigger threshold 41 (hereinafter also called trigger level). A peak exceeding the trigger threshold 41 is called an echo pulse (or echo for short). The time of such event (hereinafter called trigger time) is stored and such event is called a trigger event 23. Trigger time of an echo and the distance of the object that gave rise to this particular echo are connected via the velocity of the electromagnetic radiation emitted by the 3D sensor which can be taken as constant. Hence, trigger time and object distance are equivalent and can both be used for the horizontal axis of the diagram of FIG. 2 (this applies also to the diagrams of FIG. 3 to 6). As shown in FIG. 2 the trigger level 41 may be time-dependent, i.e. decreasing with time, to compensate for signal decrease with distance. As a result, undesired events, in particular caused by fog and snow are suppressed. Those signals are often much smaller than signals of hard targets.
In the example of FIG. 2, a so-called rising edge trigger is used, i.e. the trigger time is defined by the time the signal strength reaches the trigger level on the rising edge of the echo pulse. There are numerous other well-known trigger methods in order to determine an echo pulse and to derive a trigger time there from. Simple methods are comparators and Schmitt triggers where the signal train is compared to trigger thresholds and the time of the rising or the falling edge of the threshold crossing is recorded. More sophisticated methods are pulse width triggers and constant fraction discriminators [GED68]. A trigger condition typically contains one or more trigger parameters, like threshold, rise time, etc. The variety of conditions can be extended by combining or cascading trigger methods and parameters. In this manner trigger conditions can be tailored for expected patterns of the signal train.
Additional measures are known in order to dismiss undesired echoes and thus limiting the number of echoes to be processed. They may stem from dust, fog, etc. in the vicinity of the sensor. A frequently used technique to avoid such undesired trigger events is to use the so-called range gating. Echoes arriving from distances smaller than a minimum distance (hereinafter: the near range distance) are suppressed thus creating a blind zone in the direct vicinity of the 3D sensor.
The echoes of the signal train 21 which are received from outside said blind zone and that are fulfilling the trigger condition (e.g. that are above the trigger threshold) are intuitively labeled 1st echo pulse, 2nd echo pulse, 3rd echo pulse, last echo pulse, etc. according to their rank in the chronological order in which the echo pulses are received. It has been reported that up to 16 of these echo pulses are being processed [TRI13]. Here, too, computing power limits the number of echo pulses to be processed. Interface bandwidth and the need for real-time processing are highly limiting factors, especially in airborne applications. When the correlation between different echoes is investigated the required processing power increases tremendously with the number of returns processed. As a consequence, frequently two echoes are used [SEI08]. They are typically an acceptable compromise between the need of information of more than one pulse and the available computing performance. The present invention applies this two-echo approach.
A known two-echo approach [STE15] that exploits first and second echo uses a so-called delta range gate: One defines a so-called blind zone immediately behind (as seen from the 3D sensor) the object that caused the first echo. Any trigger event caused by an echo from objects within the blind zone (in the time domain this corresponds to a certain time slot after arrival of the first echo), will be ignored. Hence, the second trigger event can only be caused by an echo arriving from distances behind the blind zone.
The less echoes that are processed, the more crucial it is to choose those echoes for further processing that contain the most useful information. For two echoes the selection of the echoes is a central task. The selected echoes shall produce measurement distances of real objects instead of artifacts. In the context of obstacle warning under normal weather conditions, processing of the 1st echo is inevitable. In most cases it contains the echo of the actual obstacles. Thus, for 2-echo systems the remaining task of selection of echoes is limited to the selection of the remaining secondary echo. Commonly, a 1st/2nd echo scheme or a 1st/last echo scheme is used. The second echo seems to be the best choice for obstacle detection in most DVE situations. Here, the first echo might be an echo from an obscurant e.g. dust, rain, fog or snow and the second echo might stem from an actual obstacle. Ground detection is another key function. It is the main task during lift off, approach and landing and in addition, it is a significant side functionality in obstacle warning. For ground detection the last echo is to be preferred as ground will always cause the very last echo.
Hence, in conventional 2-echo systems important information is missing. Even global switching between 1st/last and 1st/2nd echo will not overcome the problem as all three echoes are useful in one and the same particular flight situation. For use of a sensor as obstacle warning system and landing aid in DVE situations, it would be very helpful to have a last echo in the short range and a second echo in the long range independently for each single shot measurement, i.e., each pixel. This requirement seems to be contradictory.
Hence, it is the object of the invention to provide a method for processing echo pulses by which this requirement can be fulfilled.
The present invention provides a method for processing echo pulses by an active 3D sensor in order to provide distance measurements of the surroundings in front of the 3D sensor, comprising the following steps:                defining a near range distance from the 3D sensor,        defining a last echo distance from the 3D sensor greater than the predefined near range distance,        receiving echo pulses of the signal emitted by the active 3D sensor and subjecting that sequence of echo pulses to a pre-defined trigger condition so that only those echo pulses are taken into consideration which fulfill the predefined trigger condition, and determining the respective trigger times and corresponding distances,        suppressing echo pulses from distances smaller than the predefined near range distance,        out of the echo pulses that fulfill the predefined trigger condition and are received from distances greater than the predefined near range distance, determining two particular echo pulses, designated first and adaptive echo pulse, according to the following procedure:        determining the echo pulse that was received first (in the following called “first echo pulse”),        determining another echo pulse (in the following called “adaptive echo pulse”) as follows:        if one or more echo pulses received from distances greater than the predefined last echo distance occur, the one of these pulses which was received first will be selected as the adaptive echo pulse,        if there are no echo pulses received from distances greater than the predefined last echo distance, then the last echo pulse received will be selected as the first echo pulse,        using the two so identified echo pulses (first echo pulse and adaptive echo pulse) for providing distance measurements in the surroundings in front of the 3D sensor.        
By using the method according to the invention, the two echoes of the signal train that are of highest relevance are selected.
The central drawback of known two echo systems is overcome: The adaptive echo pulse has the features of a last echo in the short range and the features of second echo in the long range. It shares the advantages of the 1st/2nd and 1st/last echo scheme but not their disadvantages.
The method according to the invention is applicable for all types of 3D measuring devices with time of flight measurements and subsequent pulse analysis resulting in two discrete values for echo distances.
The method according to the invention satisfies the requirements for a wide range of 3D sensor applications and operational situations, especially for optical 3D scanners. It can be completely implemented in the analogue pulse analyzer of the 3D sensor. The described logical operations performed on the signal train are easy to implement in hardware or firmware.
It is a further advantage of the method according to the invention that the filtering process according to the invention is performed at an early stage of the complete processing chain where it is most effective. The invention reduces the information of each single 3D measurement to discrete distance values that contain the desired information and that are affected by artifacts as little as possible.
Preferably, the trigger condition can be constructed in a time-dependent fashion in order to take into account that the received signal intensity and shape is highly dependent from distance and from the origin of the signal.
Preferably, the identification of first and adaptive echo pulse is conducted by applying different trigger conditions in separate process channels. That means reception and recording of trigger times of the echo pulses is done in parallel in separate pulse analyzers using different trigger conditions (e.g. different trigger thresholds). The different trigger conditions can be adapted to the typical reception conditions of first echo and adaptive echo. For instance, when the trigger condition is based on a trigger threshold, the threshold for determining the first echo will generally be higher than the threshold for the determination of the adaptive echo. In a preferred embodiment, the trigger condition in both channels can be time dependent. Another advantageous option is to apply a time dependent trigger condition only for the determination of the first echo and a constant trigger condition for the determination of the adaptive echo.
It is an important advantage that the described evaluation of first and adaptive echo by using different trigger conditions allows independent optimization with respect to the actual purpose of the two echoes. One important special case is ground detection under brownout conditions. Here, the intensity reflected from the cloud of dust may be strong and the transmitted intensity may be highly attenuated even in short distances. Therefore, a stringent trigger condition (e.g. a relatively high trigger level) is only advantageous with respect to the suppression of undesired trigger events for a first echo and a second echo but not for a last echo.
Furthermore, the use of two independent trigger conditions allows qualitative conclusions about echo signal strength and shape. For instance, the fact that an echo has fulfilled both trigger conditions may provide useful additional information. In cases where there is only an adaptive echo but no first echo or in cases where the trigger time of the adaptive echo is earlier compared to the trigger time of the first echo, the information is already inherently included in the distance information. In the important case when the trigger level for the adaptive echo is lower than that for the first echo the adaptive echo must be a weak echo.
The 3D sensor data in the form of first and adaptive echo can be further processed according to well-known techniques. In particular the 3D data can be fused with navigational data in order to provide geo-referenced 3D data of the surroundings. Filtering operations can be conducted to remove the adverse impact of sun, cloud, dust, drop-in etc. on the 3D data. Further, a classification of objects can be performed and warnings and visualization outputs can be generated and brought to the attention of the human operator of the vehicle.